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Abstract: Virtual collaboration continues to gain in significance and is attracting attention also as virtual collaborative 
learning (VCL) in education. This paper addresses aspects of VCL that we identified as critical in a series of courses named 
'Net Economy': (1) technical infrastructure, (2) motivation and collaboration, and (3) assessment and evaluation. Net 
Economy is an international online setting, focusing on the business impact of new technologies and is highly notable for 
the divergent educational and cultural backgrounds of its participants. Having been subject to research from the onset in 
2008, in which approximately 10 students were analysed and evaluated, the course has continued to gain significant 
success as a learning tool, with over 150 students currently enrolled throughout the various course cycles. In this paper we 
focus on how we implemented changes with regard to the above mentioned critical elements as part of canonical action 
research between the last course cycles. We outline the general learning scenario behind our VCL-courses, describe 
problems that we identified with the help of evaluation results and explain solution approaches and the impact of their 
implementation. The paper aims to provide a comprehensive example for virtual collaborative learning as well as 
explaining and exemplifying a systematic approach of improving complex e-learning settings through a series of steps, 
developed to ease the transition between each stage. 

Keywords: Social Networking Services, Virtual Collaborative Learning, Virtual Team Work, Web 2.0, International 
Cooperation, Community of Inquiry Framework (Col) 

1. Introduction and action research approach 

The possibilities for virtual collaboration are increasingly becoming an important aspect in theory and practice 
(Chen et al. 2008; Nunamaker et al. 2009). They are therefore being addressed more frequently in higher 
education, especially in the context of e-learning as virtual collaborative learning (VCL) (Ehsan et al. 2008; 
Hasan and Ali 2007; Lee et al. 2006; Rambe 2012; Stahl et al. 2006). Apart from the benefits of increased 
flexibility in terms of time and location and the greater focus on dealing with the topics in compliance with 
individual needs, the potential to foster communicative, social and media competencies is particularly worth 
mentioning. VCL-settings thus aim at the skills that are required and considered indispensable within a virtual 
collaboration involving heterogeneous teams working at various international locations. 

These large and growing potentials can only be realized through a suitable design and a well-structured 
learning process that creates adequate solutions to major challenges of VCL, such as social, technical and 
didactic requirements (Garrison 2011; Lee et al. 2006; Stahl et al. 2006). The contribution of this paper 
demonstrates our own VCL-activities in a series of courses between 2008 and 2012. The paper aims to provide 
a comprehensive example for virtual collaborative learning, aiming at supporting other instructors who are 
working on VCL-settings with ideas and experiences. Considering the complexity and individuality of such 
settings, the second objective is to explain and exemplify canonical action research (CAR) as a systematic 
research and improvement approach in the e-learning context. 

After briefly introducing the concept of virtual collaborative learning in Section 2.1, we will provide the 
example of the 'Net Economy' learning scenario as our own international setting for virtual collaborative 
learning in Section 2.2. Following Weber and Abuhamdieh (2011), we thereby conceive a learning scenario as a 
model for a certain type of class (Weber and Abuhamdieh 2011). In this sense the learning scenario, Net 
Economy, depicts the main processes and organizational solutions for our VCL-courses, examples include; the 
basic order of events, the time frame, approaches for video conferencing, team composition, course 
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evaluation, and assessment. Figure 1 depicts the 'build-and-evaluate' relationship between the general 
learning scenario, Net Economy, and the specific course cycles that we will focus on in this paper. 


time 



Based on our experiences and evaluation results from the Net Economy course cycle of 2008/2009 and in 
reference to the relevant literature, we will identify a selection of required improvements for the Net Economy 
learning scenario in Section 2.3. In Section 3, these needs for improvement are then compared to specific 
solution approaches, which we implemented in the course cycles of 2010/2011 and 2011/2012. After 
describing the solution approaches we will then evaluate and reflect on their effects with the help of the initial 
evaluation results from the latest course cycles. In summary, the paper addresses the following guiding 
questions: 

• What are the main characteristics of virtual collaborative learning in general as well as within our 
specific VCL learning scenario, Net Economy, and what problems arose when we accomplished virtual 
collaborative learning based on the described learning scenario? (Diagnosis) 

• What solution approaches for these problems did we derive from our own evaluation results and 
from VCL-related literature? (Action Planning) 

• How did we implement these solution approaches as interventions in the course cycles of 2010/2011 
and 2011/2012? (Intervention) 

• What were the effects of these changes? (Evaluation) 

• Is there a need for further changes in our learning scenario and what lessons have we learned? 
(Reflection) 

In answering these questions we follow the five steps of canonical action research (CAR): diagnosis, action 
planning, intervention, evaluation, and reflection (Davison et at. 2004; Susman and Evered 1978). Action 
research is committed to "[...] the production of new knowledge through the seeking of solutions or 
improvements to "real-life" problem situations." (McKay and Marshall 2001) It thus serves both, research and 
practice by studying the real world while taking a particular theoretical framework into consideration. From 
the various forms of action research that differ regarding their process model, structure, researcher 
involvement, and primary goals, we chose canonical action research (CAR), which stands out due to its 
iterative, rigorous and collaborative approach focusing on the above mentioned dual aim of improving practice 
and contributing to scholarly knowledge. It applies a cyclical process model, provokes deliberate self¬ 
involvement of the researcher, and - if carried out correctly - produces research that is both rigorous and 
relevant. Despite its practical aims, many researchers insist that action research requires a clear theoretical 
framework (McKay and Marshall 2001; Davison et al. 2004). In addition to direct references on VCL (e.g. 
Dillenbourg 1999, Prasolova-Fprland and Divitini 2002, or Tan and Lin 2008) and with regard to the VCL- 
characteristics described in Section 2.1, we therefore study VCL in the context of the Community of Inquiry 
framework for e-learning (Garrison 2011); we also consider the challenges and principles of successful (virtual) 
team work (Nunamaker et al. 2009). We specifically refer to this scholarly framework when discussing the 
problems which we identified regarding the course cycle of 2008/2009, in order to substantiate our solution 
approaches. 
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2. Virtual collaborative learning and the net economy learning scenario 
2.1 Virtual Collaborative Learning 

In the respective literature, VCL is employed in the context of various forms and aspects of learning scenarios, 
with the term itself not being used uniformly. For example "Collaborative Virtual Learning" (CVL) is also used 
by various authors (Prasolova-Fprland and Divitini 2002; Tan and Lin 2008). VCL aims at establishing guided 
collaboration in a virtual environment to enable learning. Flaythornthwaite and Andrews (2011) argue that, the 
value of merging collaboration and community in e-learning, lies in the intellectual and human benefits of 
collaborative activity. They state that such settings combine attention to work and work goals, with the 
affiliate needs of those who work together to achieve these goals (Haythornthwaite and Andrews 2011). 

With regard to the remainder of this paper we would like to point out the following characteristics of VCL as 
discussed in (Balazs 2005; Dillenbourg 1999; Ghaoui 2003; Haythornthwaite and Andrews 2011; Li and Gong 
2007; Prasolova-Fprland and Divitini 2002; Schoop et al. 2005): 

1. VCL builds on team work in small groups. Accordingly, various (educational) methods of group 
learning need to be adapted for VCL scenarios. This also implies the need for a very clear definition of 
appropriate tasks, which are usually based on authentic questions and problems from (professional) 
practice. 

2. Virtual collaborative learning harnesses information and communication technology to facilitate 
direct interaction of learners in a virtual environment, rather than in face-to-face meetings. It thus 
combines the strength of interactive learning with technology driven flexibility. The preconditions of 
virtual interaction are however significantly different from face-to-face interaction, therefore 
deliberate attention to participant interaction must be paid. 

3. Due to the high organizational challenges of VCL-settings, they require a very detailed and systematic 
preparation of the learning experience, as well as an explicit definition of the learning objectives. 

4. The complex processes, together with the active role of the learners, require strong guidance of the 
students. This special need for "teaching presence", as Garrison (2011) calls it, must be considered in 
the planning and design of the virtual and the collaborative elements of a VCL scenario. 

The theoretical background for our action research is based on the Community of Inquiry Framework (Col) 
with its collaborative-constructivist perspective on learning (Garrison 2011). The Col-framework strongly 
supports the idea of virtual collaborative learning settings, since it focuses on the opportunities of technology 
enabled learning. It is at the same time based on the premise "that a community of learners is an essential, 
core element of an educational experience when higher-order learning is the desired learning outcome" 
(Garrison 2011, p. 19). E-learning is thus considered as a means of facilitating interactivity and creating a 
quality learning experience. This corresponds with the idea of VCL in general and the Net Economy learning 
scenario in particular (see Section 2.2). The Col-framework relies upon the three interdependent elements of 
(1) social presence, (2) cognitive presence, and (3) teaching presence as constituent parts of a successful (e- 
learning) experience (Akyol and Garrison 2008; Rambe 2012): 

• The social presence is "the ability of participants to identify with a group, communicate purposefully 
in a trusting environment, and develop personal and affective relationships progressively by way of 
projecting their individual personalities" (Garrison 2011, p. 23). 

• The cognitive presence in contrast is "the extent to which learners are able to construct and confirm 
meaning through sustained reflection and discourse in a critical community of inquiry" (Garrison et 
al. 2001, p. 11). 

• Finally, the teaching presence is "the design, facilitation and direction of cognitive and social 
processes for the purpose of realizing personally meaningful and educationally worthwhile learning 
outcomes" (Anderson et al. 2001). It is this teaching presence that combines and balances the 
elements of a community of inquiry in the overall setting. 

The Col-framework serves as orientation when identifying any need for improvement as well as developing 
changes accordingly, in order to strengthen our VCL-setting Net Economy. For example, in Sections 3.1.1 and 
3.1.2 we describe a new type of coordination platform that we implemented (a social network instead of a 
traditional learning management system) and a modified team building approach that we developed, both 
with regard to the need for social presence. While this paper is meant to provide an overview of these 
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changes, their effects, for example the social effects of introducing a social networking service as the main 
coordination platform, are analysed in related papers. (Reference will be added after the review process) 

With regard to virtual teamwork as an integral part of VCL, we also consider the challenges and principles of 
effective virtual teamwork as summarized by Nunamaker et al. (2009) as guidance. Nunamaker et al (2009) 
emphasize competing demands for attention, ambiguity of virtual communication, the establishment of 
personal relationships, and the need for accessible, stable, and user-friendly technology as the biggest 
challenges for successful virtual teamwork. Table 1 summarizes the challenges and the principles that proved 
to be a good 'fit' for our VCL-setting Net Economy. 


Challenges facing virtual teamwork 

Principles for effective virtual teamwork 

• Loss of many non-verbal cues 

• Realign reward structures for virtual teams 

• Reduced mechanisms for informal 

• Find new ways to focus attention on task 

conversation 

• Design activities that cause people to get to 

• Reduced opportunities to build friendships 

know each other 

• Time zone differences 

• Build a virtual presence 

• Complicated, unreliable technology 

• Agree on standards and terminology 

• Building consensus at a distance 

• Leverage anonymity when appropriate 

• Establishing shared meaning at a distance 

• Be more explicit 

• Different work processes 

• Train teams to self-facilitate 

• Different cultures 

• Embed collaboration technology into everyday 


work 


Table 1: Virtual Teamwork - Challenges and Principles (Nunamaker et al. 2009) 


2.2 The Learning Scenario Net Economy 

In the framework of an international learning network we have offered cross-location VCL-courses entitled Net 
Economy numerous times over the past few semesters. Our general VCL-setting targets participants with 
heterogeneous educational backgrounds in the fields of business and economics, business informatics, and 
educational sciences, as well as with different cultural backgrounds from Germany (Bochum, Berlin, Dresden, 
Soest), Turkey (Istanbul), China (Shanghai), Lithuania (Kaunas), Latvia (Riga), and Indonesia (Jakarta). All 
partners agreed on the general learning scenario Net Economy, leading to a stabile VCL-concept that is being 
systematically improved throughout the course of the, previously mentioned, canonical action research 
approach. Through each course-cycle a different set of partners participated, with a different location taking 
the lead and responsibility for the course coordination. 

The general VCL learning scenario Net Economy is divided into the two phases 'production phase' and 
'experience phase'. Throughout the course, project work is conducted in small teams of 4-6 students across 
various locations, with team composition as well as presentation and discussion of findings being conducted 
through video conferencing. By separating these phases, the learning and working process is structured as a 
project with the use of predefined milestones. The students are asked to present and discuss their (interim) 
findings at so-called 'steering committees' and within phase-specific final presentations. These steering 
committees and final presentations are held at each location and are merged together through video 
conferencing. The participants are prepared for this through introductory presentation training, but also 
through e-lectures on project management, group coordination, and the use of specific communication tools 
(e.g. the video conferencing tool 'Adobe Connect'). These efforts aim to reconcile the differing levels of media 
and teamwork skills among the students and enable them to fulfil the requirements of the VCL-setting. 

During the production phase participants experience a "learning through teaching approach" (Biswas et al. 
2005). Under their instructors' guidance they develop multimedia learning materials such as web-based 
training systems or Google sites on methods of strategic management (VRIO, business model analysis, SWOT 
analysis, scenario planning) or current IT-topics such as "Web 2.0". In this way they are able to gain a new 
insight and prepare themselves for the next phase. The experience phase, in contrast provides the students 
with a case study, with previous examples including; electronic marketplaces (2008/2009), grocery home 
delivery (2010/2011), or the use of Web 2.0 within and between companies (2011/2012). While working on 
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these case studies the students can apply their newly acquired knowledge to real-life business challenges. The 
two phases are thus linked together in a way that the learning material, created during the production phase, 
provides a substantive knowledge base for the tasks within the case study phase. In addition, both phases 
(production and experience phase) provide the participants with the opportunity to foster international 
contacts and to gain experience in cross-cultural, technology-based learning and work. 

Figure 2 summarizes the main aspects of the learning scenario Net Economy: its global context, the use of 
information and communication technology (especially Web 2.0 applications), its two phases, the student 
teams, as well as the combination of classroom presentations and online group-work phases. Further details 
on the organizational structure and the main processes of the Net Economy classes will be provided in the 
following Sections, where we will discuss problems that have arisen and the solutions that we developed with 
respect to these problems. 


Podcasts 




Students 

Int. Student Teams 


Social 

Networks 


Wikis 


Social 
\EtoOkmarks 



Tag 

Clouds 


Rich Media 
Communication 


Syndication 


E Classroom Presentation (int. Online-Meeting) □ Online Phase (group work phase) 

Figure 2: The Net Economy Learning Scenario 


In summary, the learning scenario Net Economy is not only associated to cognitive, but also to affective and 
psychomotor learning goals. Considering the mostly, content-focused, classes in the students' curriculums, it 
was developed to help the students experience coordination, team work, language and cultural challenges in a 
modern, technology-based work environment. In addition to the impartment of new knowledge, attention is 
paid to fostering and deepening vocational abilities, such as the use of new media (software tools, Web 2.0 
applications) and ultimately the preparation for life-long learning (Chen et al. 2008; Safran et at. 2007). Due to 
the international setting, the students understand and fully agree with the need for virtual collaboration and 
the use of English as the primary language for correspondence. In addition, the cooperation of the different 
universities and their instructors/researchers created an agile and innovative teaching/research alliance within 
the e-learning field itself. 

The learning scenario Net Economy as a VCL-setting aims at initiating: 

■ Online group-work processes; 

■ The employment of methods of strategic management, project management, and team coordination; 

■ The design of social, medial and cultural activities and their interactions in a global and digitized context; 

■ An on-going creative cooperation and exchange among the participating institutions. 


The Net Economy course cycles which we derived from this general learning scenario have proven to be a 
motivating and enlightening learning experience for all those involved. In the courses, e-learning and online 
cooperation were not just phase-specific or redundantly supporting features of a traditional setting, but rather 
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provided the systematic and consistent foundation for innovative learning and working processes. Still, the 
challenging overall setting incurred problems and the evaluation results regularly point at improvement needs. 
In Section 2.3 we will show, which problems we identified after the course cycle of 2008/2009 and how these 
challenges relate to the above described scholarly framework. 

2.3 Necessary Improvements 

Our evaluation of the 2008/2009 course cycle focused on learning satisfaction and was accomplished in three 
steps via online forms: (1) At the beginning of the course the students' attitudes and expectations were 
surveyed with the help of a master data sheet. (2) During the experience phase, the students were asked to 
evaluate the learning material and the case study provided to them. (3) Finally, after the course ended, the 
students were asked to evaluate the complete course and aspects of the instructional setting, e.g. the 
collaboration concept. The evaluated items of this final questionnaire covered five subject areas that score a 
satisfaction rating (scale: 1 = very good,..., 6 = inadequate) for key aspects of the learning scenario: 

• Overall learning arrangement; 

• Support by lecturers and tutors; 

• Organization and implementation of project work; 

• International collaboration; 

• Technical infrastructure / Use of media. 

While systematically building on to this rather simple evaluation approach during the past years (see Section 
3.3), we kept the questions from 2008/2009 in the questionnaire to allow for a comparison of the various 
course cycles regarding the students' learning satisfaction. Figure 3 summarizes the results from the last three 
runs of the class. 



While the students from 2008/2009 were very satisfied with their support, the good evaluation results for the 
overall course and for the organization of the project work leave leeway for further improvements. According 
to the students' feedback, the international collaboration and the technical infrastructure provided to them 
turned out to be the most challenging aspects of the setting. The depicted evaluation results for the later 
course cycles show that the changes we implemented in 2010/2011 and 2011/2012 had positive effects 
regarding these two critical aspects of the setting. Table 2 shows the detailed results from the 2008/2009 
evaluation. The evaluation combined a standard form, provided by one of the participating universities, with 
additional questions concerning the specific Net Economy setting. Since grading questions were added, two 
additional measurement scales needed to be included in the evaluation: *: approval from 1 to 4 (1 = absolutely 
yes ... 4 = absolutely no) and **: grades according to the German grading system (1 = very good ... 6 = 
inadequate) 
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Aspect Statement 

Av. 

DV 

N 


The course was well structured.* 

1.93 

0.65 

29 


1 gathered knowledge and skills of high relevance.* 

1.89 

0.74 

28 


Theoretical and practical aspects were combined successfully.* 

1.82 

0.61 

28 


There was a clear conception that was kept throughout the course.* 

1.89 

0.75 

28 


1 enjoyed the inclusion of external experts.* 

1.48 

0.59 

25 

to 

Blackboard (Learning Management System) supported the learning process* 

1.96 

0.92 

28 

CU 

U 

Overall grade for the course** 

2.18 

0.86 

28 


The instructors were well prepared.* 

1.36 

0.68 

28 

to 

The instructors lead the course in a clear manner.* 

1.75 

0.75 

28 

O 

4-» 

The instructors demonstrated dedication to the course.* 

1.36 

0.68 

28 

3 

l. 

The instructors were open to questions/criticism.* 

1.54 

0.74 

28 

to 

C 

Overall grade for the instructors** 

1.61 

0.68 

28 


1 valued the project structure of the course. * 

1.96 

0.76 

27 


There was a clear structure of project phases due to the steering committees.* 

1.93 

0.73 

27 

o 

The online presentations of (intermediary) results were an important exercise. 

* 

1.89 

0.80 

27 

u 

0^ 

The project tasks were clear and unmistakable.* 

2.22 

0.89 

27 

o 

&- 

a. 

Overall grade for the project organization** 

2.22 

0.98 

27 


Online collaboration with students from other locations was a good 
experience.* 

1.37 

0.69 

27 

c 

o 

■4— » 

Online teamwork helped me to get introduced to helpful tools and procedures. 

* 

1.89 

0.85 

27 

fu 

O 

The assignment of given roles facilitated our teamwork. * 

2.27 

1.00 

26 

-Q 

_ru 

1 enjoyed the online collaboration.* 

1.81 

0.94 

26 

"o 

u 

Overall grade for the online collaboration** 

2.37 

1.18 

27 

+■» 

Male / female 

16/12 

- 

28 

ru 

Q. 

Age 

25.19 

2.93 

26 

u 

+3 

1 enjoyed the topics of the course.* 

1.93 

0.77 

28 

ru 

Q- 

My attitude towards e-learning developed positively.* 

1.96 

0.70 

27 


Table 2: Net Economy evaluation results 2008/2009 

*: Approval from 1 to 4 (1 = absolutely yes ... 4 = absolutely no) 

**: Grades according to the German grading system (1 = very good ... 6 = inadequate) 

Taking both the presented scholarly framework and our own evaluation-based experiences into consideration, 
we will now describe the need for improvements that we derived in preparation for the 2010/2011 course 
cycle. We make no claim that this selection is complete, rather we chose aspects that came to the fore when 
discussing our experiences and the evaluation results in preparation of an extended and improved learning 
scenario. 

1. Flexible, low-cost and technically simple infrastructure for plenary interactions and collaborative 
learning and working processes within the teams: As Nunamaker et al. (2009) point out, 
"Technological glitches will cripple the productivity of even the most knowledgeable and motivated 
virtual teams." In line with this statement, during the 2008/2009 course we encountered considerable 
difficulties due to differing technical equipment and skills among the participating students, 
instructors and institutions. A low-cost and flexible technical architecture which is geared towards the 
interests and abilities of the participants therefore appears more important than sophisticated 
functionality of high end tools. The challenge lies in providing a coherent package of communication 
tools in an integrated form which fosters effective team-work processes and positive team 
development, while being embedded into the everyday life of the students (see Table 1; Nunamaker 
et al. 2009). Also, the technical infrastructure (especially for the purpose of video conferencing) needs 
to be simple in order to allow for the flexible integration of all participating institutions and 
instructors with their heterogeneous equipment and skills. With regard to group dynamics, video 
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conferencing should be made available within the teams, as this influences the team cohesion more 
positively than other forms of communication (Ehsan et at. 2008). 

2. Team composition, work-load distribution within the teams and assessment: In the evaluation of 
2008/2009, on a scale from 1 to 4 (1 = I absolutely agree, 4 = I absolutely disagree) students expressed 
a very positive attitude towards working with students from other locations and disciplines ("Online 
collaboration with students from other locations and disciplines was a significant experience for me": 
1.37). They also attested that they enjoyed online collaboration in general ("I enjoyed the online 
collaboration": 1.81). Against this background, the overall grade for the experienced online 
collaboration (2.37 on a scale from 1 = very good to 6 = inadequate) and its considerable deviation 
from the standard (1.18) indicate that factors with a negative influence exist which require further 
investigation. Students were allowed to comment on their responses, which revealed that some of 
them experienced the collaboration processes as too challenging and exhausting. Several students 
complained about freeloaders who made the work in their teams more complicated. In 
correspondence with these experiences concerning the social presence within the Community of 
Inquiry (Garrison 2011), several authors point out and stress the relevance of team and member- 
related factors such as team composition and internal heterogeneity, team spirit or the consequences 
of a free-rider mentality (Liccardi et al. 2007; Ehsan et al. 2008; Hasan and Ali 2007). In 2008/2009 the 
students only had little influence on the team composition. When the groups were compiled, merely 
brief profiles and role requests that students had to fill out before the course started, were taken into 
account by the instructors. The students, however, took little advantage of this possibility and in most 
cases did not provide any convincing profiles. It is therefore necessary to develop a team building 
approach that boosts social presence in the setting in order to facilitate the development of team 
spirit and a distinct sense of responsibility among the team members. With the emphasis lying on 
virtual team work and the desired complex and interdisciplinary tasks, the modus operandi should 
provide a heterogeneous composition of the teams. Apart from this composition, the quoted factors 
and framework conditions for the setting must be taken into account in the assessment and 
particularly in the grading process. The team results should be given priority, but at the same time, 
individual differences in performance need to be considered to reduce the negative impact of 
freeloaders. This aspect refers to the need of a teaching presence as part of the Col-framework. 

When looking at the solution approaches for theses conceptual problems of the Net Economy learning 
scenario, the following framework conditions regarding the setting as part of an international learning network 
need to be kept in mind: 

3 Limitations regarding language abilities of students and instructors range from minor differences in 
the level of fluency, up to language incompatibilities. 

4 Various ways of establishing the course as part of a broader curriculum imply major differences 
regarding the motivation and commitment of the participants at the different locations. 

5 The student teams are characterized by extensive heterogeneity with regard to the participants' 
nationalities and educational backgrounds. 

6 There are considerable differences between the participants regarding their technical equipment and 
media experience. 

7 There are also considerable differences between the instructors as to their knowledge and experience 
regarding the learning scenario and the course topics. 

Table 3 summarizes the identified problem areas and assigns them to the build-and-evaluate loop according to 
our canonical action research approach. In Section 3 we will present the solution approaches listed in the 
column on the right of the table. 
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Phase 

Problem Areas 


Solution Approaches | 

Build 





3.1 Motivation and Identification 



o 

c 


- Technical Infrastructure: NING 


d) 

D 

4—< 

rtJ 

u 

-O .2 
c ■*-* 

5 E 


- Self-responsible Team Building Process 


U 





Realize 

3 

4-» 
l ft 

E 

M— 

c 

c 

CD 

TD 

T3 

O 

CO 

K o 
u 

3.2 Collaboration Support 

- Technical Infrastructure: Web 2.0 

Evaluate 

3 

’c 

JO 

u 

|2 

c 

o 

4-* 

(TJ 

> 

4-> 

o 

2 


ssessment/ 

Valuation 

3.3 Assessment and Evaluation 

- Assessment and Grading 

- Competence based Evaluation 





< 



Table 3: Problem areas and solution approaches 

3. Solution approaches 

In 2008/2009, 64 students from 5 universities and 3 countries (China, Germany, and Turkey) participated in the 
Net Economy course. The following solution approaches were developed in preparation for the expanded 
version of the 2010/2011 course with 96 participants from 6 universities and 3 countries: Germany, Lithuania 
and Latvia. In 2011/2012, 180 students from 5 universities and 2 countries (Germany and Indonesia) 
participated. More than 80% of both Bachelor and Master students in all course cycles had a Business or IT- 
focus, with Master students being given additional tasks by their home universities. While the course duration 
was reduced to 10 weeks (compared to 14 weeks in 2008/2009) due to different semester schedules at the 
various locations for these course cycles, the main structure of the learning scenario as described in Section 
2.2 was retained. In contrast to 2008/2009, where only students from the German language study programs 
participated, in 2010/2011 and 2011/2012 approximately 40% of the students were from an English language 
study program, however, as of yet no native speakers have participated. 

3.1 Motivation and Identification 

As described in Section 2.3, the evaluation of 2008/2009 pointed at the existence of a relationship between 
the students' lack of involvement in the preparation of the course and problems of cooperation which later 
occurred within the teams. Based on our experience, the students' comments, and the emphasis of the need 
for a social presence in the Col-framework, we argue that these problems in particular were caused through 
some students' insufficient identification with their teams and a sluggish team building process. To deal with 
this matter, the following adjustments have been initiated: (1) change of the technical infrastructure by using 
the social networking service NING as the central course platform instead of a traditional learning 
management system (e.g. Blackboard in 2008/2009); (2) modification of the team building process; (3) 
introduction of a modified roles approach. 

3.1.1 Change of the Technical Infrastructure: NING 

The first modification concerns the central course platform of the learning scenario. The platform should 
reflect the reality of today's globalized world and media-dominated social environment, and it must meet the 
framework conditions of the VCL-setting. With regard to the identified problems, it additionally needs to be 
flexible and powerful in terms of social identification and collaboration. Rather than a classical learning 
management system (LMS) as used in the Net Economy course of 2008/2009 (Blackboard), the employment of 
a social networking service (SNS) appears more suitable for the learning scenario Net Economy with regard to 
the need of social presence. Apart from the additional functionality in terms of communication and 
collaboration, SNS facilitate identification with the course and help students to get to know each other, while 
being part of the students' day-to-day life (see table 1, Nunamaker et al. 2009). We chose NING as a platform 
to create a private SNS for the Net Economy course. NING allowed us to define unique profile questions, and 
to easily employ main SNS-features like community and group creation, moderation, use of rich media, rss 
feeds, tracking of latest activities (including status updates), and various forms of communication. The 
evaluation results for the technical infrastructure and the survey of the use of NING both in 2010/2011 and 
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2011/2012 support this idea. A detailed analysis specifically on the social effects of the changed coordination 
platform is available in. (Reference will be added after the review process) 

3.1.2 Modification of the Team Building Process 

Another crucial aspect for the success of the learning scenario needing improvement is the team building 
process. For this purpose, a two week preparation phase was added to the learning scenario in which the 
students had time to acquire the information and skills (e.g. media skills) necessary to accomplish the tasks of 
the production and experience phase. With this preliminary preparation phase the students were given the 
opportunity to familiarize themselves with the tasks and the various roles that would arise during the team 
work process. 

Whereas in 2008/2009 the composition of the team was determined by the course instructors, in this new 
approach the task of defining the team was delegated to the students during the preparation phase. Students 
had to create a personal profile on NING, which then served as their application to become a team member. 
The questions concerning the profile prompted a critical reflection on their individual strengths, weaknesses 
and previous experiences relating to both the topics of the class and virtual collaboration (e.g. use of media). A 
secondary objective of this self-reflection consisted of preparing an expanded evaluation of the development 
of skills during the course, which will be described as our enhanced competence-based evaluation approach in 
Section 3.3. Based on the profiles that the students developed, the instructors appointed the team leaders 
who then recruited their teams. The composition of the teams was thereby restricted with regard to group size 
and the number of team members from each participating location: A maximum of six students per team with 
no more than two students from each location being allowed. Shortly prior to the start of the production 
phase, conflicts were solved by the instructors, if necessary. By delegating the task of group definition we 
intended to strengthen the students' self-reliance and identification with their teams - thus, social presence. 
The team building approach turned out to be a very effective element of teaching presence, which also helped 
to establish the SNS successfully. (Reference will be added after the review process). 

3.2 Collaboration Support: Web 2.0 Applications 

In order to produce content, in 2008/2009 the students employed predetermined desktop applications which 
were not designed or suitable for the collaboration in geographically distributed teams. Among them, special 
tools for the design and the production of web-based training, e.g. Adobe Captivate and Lersus (as an editing 
tool for web-based trainings). Considering the objectives and framework conditions of the learning scenario, 
this unsustainable approach was the cause for various difficulties (usability, version control, and 
cooperation/collaboration). We therefore not only changed the course platform, but we also introduced 
various web 2.0 tools, e.g. Google docs, Google sites, blogs, wikis, project management tools, etc. during 
preparation phase. This enabled a simultaneous, straightforward online collaboration. However, with the 
exception of the central course-platform NING, the specific choice of communication and collaboration tools 
was left entirely up to the students. This was a major modification compared to 2008/2009, when the tools 
were rigidly predetermined. 

3.3 Assessment and Evaluation 

3.3.1 Assessment and Grading 

In the 2008/2009 Net Economy course grading was based on separate team assessments for the phases of the 
setting. Frustration and motivation problems arose as a result of the free-rider mentality exhibited by a small 
group of students. For a better assessment of the combined individual and team performance within the 
learning scenario, and also considering the stronger self-reliance exhibited by students in the modified setting, 
we developed and implemented the following assessment procedure: Firstly, two instructors evaluated the 
solutions presented by a team based on an evaluation sheet with scores ranging from 0 to 100 points. This 
team result accounted for 50% of the grading of each team member. Secondly, the score was multiplied by the 
number of team members and assigned to the team as a whole. For example, if a team's results were 
evaluated with 80 points (representing the grade B), then a team with 6 team members received a score of 
480 points. Based on this score the team then conducted a self-assessment. It was the team's responsibility to 
agree on a fair allocation of the score within the team by discussing individual performances and contributions 
by the team members. The result of this self-assessment accounted for another 25% of each team member's 
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grade. Thirdly, to guarantee that students not only concentrate on the findings and ideas of their own team, a 
multiple choice quiz was included after production phase. The test covered only learning material and content 
developed by the student teams during production phase and counted for another 25%. In conclusion, this 
approach integrated the individual performance of each student and the team result, while allowing the 
instructors to consider special dedication and free-rider mentality among the team members. 

3.3.2 Competence-based Evaluation 

Finally, to improve the assessment of the course impact on the participants' skills and their learning 
satisfaction, the evaluation process as described above was enhanced. We developed a competency-oriented 
evaluation approach based on an inventory by Paechter et al. (Fink 2010; Paechter et al. 2007). As shown in 
Figure 4, this approach covers seven areas: Experience with (1) use of wikis (Google Sites) (2) social 
communities, (3) international collaboration (4) group-work, (5) project work, (6) presentation of findings and 
(7) the subject area of the course. Emphasis was placed on the assessment of the development of media skills 
and collaboration / group working skills. Testing the students before and after the course determined the self- 
perceived change in the skillset. Additionally, the students were asked to rate the individual importance of 
each item. This way, in addition to the feedback on the students' learning satisfaction, we are also now able to 
assess the development of their skills. 


Competence-based evaluation"Net Economy" 2011/2012 
Experience with... 


...Wikis ...Social ... international ... group work ... project work... presentation ... the subject 
Communities collaboration of findings area of the 

course 



—♦— 2011/2012 

Pre-Test 

-■- 2011/2012 

Post-Test 

-*- 2011/2012 
Importance 


1 = Very little / 
not important 
6 = very much/ 
important 
N = 59 


Figure 4: Evaluation results: Self-assessment of acquired skills in 2011/2012 

Overall, the students recognized an increase in competence in all five areas in which high levels of skills were 
achieved. This suggests that the main goals of the learning scenario were achieved and that the modifications 
described in this paper had the desired effects regarding the overall goal of enabling learning in a Community 
of Inquiry with social, cognitive, and teacher presence. A description and discussion of the detailed results of 
the evaluation (including the open-text comments of the students) were left out in this paper in favour of a 
more comprehensive derivation of improvement needs (Section 2.3) and solution approaches (Section 3). 
Flowever, they are the subject of related work (references will be added after the review process). 

4. Summary and outlook 

This paper has addressed considerable aspects of VCL-arrangements as identified during a series of VCL- 
courses called Net Economy that we offered over the past few years. Structured according to the cyclical 
process model of canonical action research (diagnosis, action planning, intervention, evaluation, and 
reflection) we have 

■ (1) described the general learning scenario Net Economy and diagnosed the course cycle from 2008/2009; 

■ (2) identified major conceptual problems (technical infrastructure, motivation and identification, 
collaboration, and assessment / evaluation) and 
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■ (3) documented the implementation of solution approaches (use of the social network NING; use of Web 
2.0 tools instead of single-user desktop applications; deferring team building process to students; 
enhanced grading approach; enhanced evaluation approach); 

■ (4) / (5) presented some initial evaluation results that help us reflect the effects of our interventions. 

Our changes can be considered successful and have helped us improve the Net Economy learning scenario, 
specifically for the courses from 2010/2011 and 2011/2012. At the same time the depiction of our complete 
action research approach was conceived as a contribution to the scholarly discussion of virtual collaborative 
learning and how social presence, cognitive presence and teaching presence can be strengthened to create a 
Community of Inquiry and a successful learning experience. However, in this paper we compared several 
course cycles in which some of the variables have changed over time. For example, the number of students 
rose from 64 to 180 and the set of participating locations with their individual cultural characteristics changed. 
The cultural challenges of the setting in general and the effects of the above mentioned changes need to be 
addressed in future research. Additionally, the evaluation and the reflection of our interventions have been 
given only superficial attention thus far and need more emphasis in future evaluations. For example, we have 
gathered rich data on the use of the different technical infrastructures implemented, such as the new course 
platform NING and the interconnections that arose between the students using this platform, in particular 
during the 2011/2012 course cycle. Currently we are using this data in order to analyse the quality of the social 
presence that we strived for by changing from a traditional learning management system to a social network. 
Findings indicate that NING led to a well-meshed network of relationships among the students from the 
participating locations and that the development of these relationships was fundamentally influenced by the 
phase concept and team building approach of the Net Economy learning scenario, described in the paper. NING 
was actively used throughout the complete setting for both; class related tasks and pure social matters. For a 
detailed analysis see (reference will be added after review). Finally, we have not paid much attention to the 
need for cognitive presence in this paper. In order to address this important aspect we need to further change 
the communication infrastructure of the class in a way that the communication channels of the students 
become more transparent and accessible for us as researchers also outside the social network. Despite these 
and other limitations, we consider the 2010/2011 and 2011/2012 Net Economy courses as further successful 
evolutionary steps in our cyclical canonical action research approach. We will now start the analysis and 
discussion of the results from the latest course cycle and the interventions that we developed in preparation 
for this latest run. 
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